Revisiones sistemáticas en cinco pasos: V. Cómo interpretar la evidencia / Systematic reviews in five steps: V. Interpreting the findings

El último paso en una revisión sistemática es interpretar la evidencia de forma que pueda ser útil para sustentar la toma de resultados. Es importante presentar los resultados de forma claramente diferenciada, asignar el nivel de fuerza de la evidencia que respalda cada uno de ellos teniendo en cuenta factores como el diseño de los estudios, su calidad metodológica y el sesgo de publicación. También es necesario considerar las variaciones en la fuerza de la asociación. El objetivo es detectar en qué tipo de pacientes la intervención analizada será más o menos efectiva, tendrá mayor o menor impacto una exposición determinada, o será de mayor utilidad una prueba diagnóstica. En esta fase, para facilitar la interpretación de los resultados, se puede añadir una estimación de la magnitud de la asociación, bien global o bien estratificada en función de las características de los participantes. Toda esta información ayudará a la hora de formular recomendaciones prácticas con utilidad clínica. (AU)

The last step in a systematic review is the interpretation of the findings. The important findings need to be explicitly identified. A level of strength of evidence should be assigned to support each key finding, based on factors such as study design, methodological quality and risk of publication bias. Variations in the magnitude of associations observed also need to be explored. The aim of this analysis is to determine in which clinical groups the intervention is more or less effective, the impact of exposure is greater or lesser, or a diagnostic test is more useful. At this stage, for better interpretation of the findings, the magnitude of the association can be estimated either globally or stratified according to the characteristics of the participants. All this is helpful in formulating recommendations for clinical practice and policy. (AU)

[Systematic reviews in five steps: V. Interpreting the findings]. / Revisiones sistemáticas en cinco pasos: V. Cómo interpretar la evidencia.

The last step in a systematic review is the interpretation of the findings. The important findings need to be explicitly identified. A level of strength of evidence should be assigned to support each key finding, based on factors such as study design, methodological quality and risk of publication bias. Variations in the magnitude of associations observed also need to be explored. The aim of this analysis is to determine in which clinical groups the intervention is more or less effective, the impact of exposure is greater or lesser, or a diagnostic test is more useful. At this stage, for better interpretation of the findings, the magnitude of the association can be estimated either globally or stratified according to the characteristics of the participants. All this is helpful in formulating recommendations for clinical practice and policy.

Aplicabilidad de modelos predictivos de falla renal en enfermedad renal crónica: una revisión de alcance / Usability of predictive models of kidney failure in chronic kidney disease: a scoping review

Resumen Introducción: la identificación de los pacientes con mayor riesgo de progresar a falla renal es fundamental para la planeación del tratamiento en la enfermedad renal crónica, pero no ha podido llevarse a cabo consistentemente. Los modelos de predicción podrían ser una herramienta útil, sin embargo, su usabilidad en la Enfermedad Renal Crónica es limitada hasta ahora y no se comprenden muy bien las barreras y limitaciones. Métodos: se desarrolló una revisión de alcance de la literatura disponible sobre modelos predictivos de falla renal o reglas de pronóstico en pacientes con Enfermedad Renal Crónica. Las búsquedas se realizaron sistemáticamente en Cochrane, Pubmed y Embase. Se realizó una revisión ciega e independiente por dos evaluadores para identificar estudios que informaran sobre el desarrollo, la validación o la evaluación del impacto de un modelo construido para predecir la progresión al estadio avanzado de la enfermedad renal crónica. Se realizó una evaluación crítica de la calidad de la evidencia proporcionada con el sistema GRADE (Grading of Recommendations Assessment, Development and Evaluation). Resultados: de 1279 artículos encontrados, fueron incluidos 19 estudios para la síntesis cualitativa final. La mayoría de los estudios eran primarios, con diseños observacionales retrospectivos y unos pocos correspondieron a revisiones sistemáticas. No se encontraron guías de práctica clínica. La síntesis cualitativa evidenció gran heterogeneidad en el desarrollo de los modelos, así como en el reporte de las medidas de desempeño global, la validez interna y la falta de validez externa en la mayoría de los estudios. La calificación de la evidencia arrojó una calidad global baja, con inconsistencia entre los estudios e importantes limitaciones metodológicas. Conclusiones: la mayoría de los modelos predictivos disponibles, no han sido adecuadamente validados y, por tanto, se consideran de uso limitado para evaluar el pronóstico individual del paciente con enfermedad renal crónica. Por lo tanto, se requieren esfuerzos adicionales para centrar el desarrollo e implementación de modelos predictivos en la validez externa y la usabilidad y disminuir la brecha entre la generación, la síntesis de evidencia y la toma de decisiones en el ámbito del cuidado del paciente.

Abstract Background: the identification of patients at higher risk of progressing to kidney failure is essential for treatment planning in chronic kidney disease, but it has not been possible to do this consistently. Predictive models could be a useful tool, however, their usability in chronic kidney disease is limited and the barriers and limitations are not well understood. Methods: a scoping review of the available literature on ESRD predictive models or prognostic rules in chronic kidney disease patients was developed. Searches were systematically executed on Cochrane, MEDLINE, and Embase. a blind and independent review was carried out by two evaluators to identify studies that reported on the development, validation, or impact assessment of a model constructed to predict the progression to an advanced stage of chronic kidney disease. A critical evaluation of the quality of the evidence provided with the GRADE system (Grading of Recommendations Assessment, Development and Evaluation) was made. Findings: of 1279 articles found, 19 studies were included for the final qualitative synthesis. Most of the studies were primary, with retrospective observational designs and a few corresponded to systematic reviews. No clinical practice guidelines were found. The qualitative synthesis showed high heterogeneity in the development of the models, as well as in the reporting of global performance measures, internal validity, and the lack of external validity in most of the studies. The evidence rating was of low overall quality, with inconsistency between studies and important methodological limitations. Conclusions: most of the available predictive models have not been adequately validated and, therefore, are of limited use to assess the individual prognosis of patients with chronic kidney disease. Therefore, additional efforts are required to focus the development and implementation of predictive models on external validity and usability and bridge the gap between generation, synthesis of evidence, and decision-making in the field of patient care.

Effects of experience with access regimes on stewardship behaviors of small-scale fishers.

Governance regimes that assign exclusive access to support collective action are increasingly promoted to manage common-pool resources under the premise that they foster environmental stewardship. However, experimental evidence linked to existing policies that support this premise is lacking. Overlapping access policies in small-scale fisheries provide a unique opportunity to test the effects of access regimes on users' stewardship behaviors. We performed a lab-in-the-field experiment to assess how fishers' previous experience with access regimes relates to compliance and peer enforcement (n = 120). Fishers' compliance and peer-enforcement decisions were compared in a common-pool-resource game. Treatments differed in framing to represent exclusive access and pseudo-open access regimes, both of which fishers face in real life. To contrast behavior in the game with real-life observations, we compared fishers' associations that have shown relatively high and low management performance under exclusive access policies. Compliance and peer enforcement were higher under the exclusive access treatment than under the pseudo-open access treatment only for fishers' associations with high management performance in real life. Behaviors in the game reflected differences between associations in real life. Our results support previous research on ocean governance by experimentally assessing the role of access regimes in determining users' stewardship and suggest potential mechanisms for stewardship internalization.

Efectos de la experiencia con regímenes de acceso sobre comportamientos de gestión responsable de pescadores a pequeña escala Resumen Las políticas que asignan acceso exclusivo a grupos de usuarios para apoyar la acción colectiva son cada vez más promovidas para el manejo de recursos de uso comunitario bajo la premisa de que fomentan la gestión ambiental responsable. Sin embargo, la evidencia experimental vinculada a políticas existentes que respalde esta premisa es insuficiente. La superposición de diversas políticas de acceso en las pesquerías a pequeña escala proporciona una oportunidad única para analizar los efectos de los regímenes de acceso sobre el comportamiento de gestión de los usuarios. Realizamos un experimento, llevando el laboratorio al campo, para evaluar cómo la experiencia previa de los pescadores con regímenes de acceso se relaciona con sus comporatamientos de cumplimiento y de sanción de pares (n = 120). Comparamos el cumplimiento con cutoas de extracción de los pescadores y sus decisiones de sancionar a pares que incumplian las cuotas en un juego de recursos de uso comunitario entre dos tratamientos. Los tratamientos variaban en la contextualización del juego para representar una pesquería de acceso exclusivo y una de pseudo libre acceso, a las que se enfrentan los pescadores en la vida real. Para contrastar el comportamiento en el juego con las observaciones de la vida real, comparamos los resultados de asociaciones de pescadores que han mostrado un desempeño de manejo relativamente alto y bajo con las políticas de acceso exclusivo. El cumplimiento y la sanción de pares fueron mayores bajo el tratamiento de acceso exclusivo que bajo el de pseudo libre acceso sólo para aquellas asociaciones de pescadores con un alto desempeño de manejo en la vida real. Los comportamientos en el juego reflejaron las diferencias entre las asociaciones en la vida real. Nuestros resultados respaldan investigaciones previas sobre la governanza de recursos marinos mediante la evaluación experimental del papel que tienen las políticas de acceso en la determinación de la gestión del usuario y sugieren mecanismos potenciales para la internalización de dicha gestión.

Assessment of a Multidimensional School Collective Efficacy Scale to Prevent Student Bullying: Examining Dimensionality and Measurement Invariance / Una evaluación de la Multidimensional School Collective Efficacy Scale para prevenir el acoso en escolares: el examen de la dimensionalidad y la invarianza de medida

The construct of a school collective efficacy to prevent bullying has attracted attention as a way to increase a positive, school-wide climate. The current study tested the fit of several first-order models of school collective efficacy to prevent (uni-dimensional, two-dimensional, and three-dimensional) bullying using a sample of 579 male (Mage = 14.31, SD = 1.78 years old) and 589 female (Mage = 14.56, SD = 1.83 years old) Mexican adolescents. The models were validated by the extent to which the model was invariant by gender and by educational level (secondary vs. high school). Moreover, the discriminant and concurrent validity of model dimensions were examined through their relationships with other constructs. The results suggest that school collective efficacy is a three-dimensional construct, with supporting evidence for cohesion, students’ social control, and teachers’ social control dimensions. Measurement invariance was found in this three-dimensional measurement model by gender and educational level. The latent means difference analysis showed some differences by gender and educational level on factors of school collective efficacy. Finally, results support our hypotheses related to discriminant and concurrent validity in relation to external variables. Overall, findings indicate this three-dimensional model is useful to measure adolescents’ perceptions of school collective efficacy. (AU)

El constructo eficacia escolar colectiva para la prevención del acoso escolar ha recibido una creciente atención como una forma de mejorar el clima escolar. En el presente estudio se evaluó el ajuste de varios modelos de primer orden para medir la autoeficacia escolar colectiva para la prevención del acoso escolar (unidimensional, bidimensional y tridimensional), utilizando una muestra de 579 chicos (Medad = 14.31, DS = 1.78 años) y 589 chicas (Medad = 14.56, DS = 1.83 años) mexicanos. Para validar el modelo en distintos grupos se analizó la invarianza de medida por sexo y nivel educativo (secundaria vs. bachillerato). Además, se examinó la validez discriminante y concurrente analizando la relación de la eficacia escolar colectiva con otros constructos. Los resultados indican que la eficacia escolar colectiva es un constructo tridimensional conformado por las dimensiones de cohesión y confianza, control social de los estudiantes y control social de los docentes. La evidencia constata la invarianza de medida del modelo por sexo y nivel educativo. El análisis de las diferencias de medias latentes muestra diferencias por sexo y nivel educativo en las dimensiones de la eficacia escolar colectiva. Finalmente, los resultados confirman las hipótesis relativas a las relaciones de la eficacia escolar colectiva con otros constructos. Los hallazgos sugieren que el modelo tridimensional es útil para medir la percepción adolescente de la eficacia escolar colectiva. (AU)

Reliability and External Validity of Personality Test Scores: The Role of Person and Item Error / Fiabilidad y Validez Externa de las Puntuaciones en los Tests de Personalidad: El Papel del Error en las Personas y en los Ítems

Background: This article explores the suitability of a proposed “Dual” model, in which both people and items are sources of measurement error, by assessing how the test scores are expected to behave in terms of marginal reliability and external validity when the model holds. Method: Analytical derivations are produced for predicting: (a) the impact of person and item errors in the amount of marginal reliability and external validity, as well as the occurrence of “ceiling” effects; (b) the changes in test reliability across groups with different average amounts of person error, and (c) the phenomenon of differential predictability. Two empirical studies are also used both as an illustration and as a check of the predicted results. Results: Results show that the model-based predictions agree with existing evidence as well as with basic principles in classical test theory. However, the additional inclusion of individuals as a source of error leads to new explanations and predictions. Conclusions: The proposal and results provide new sources of information in personality assessment as well as of evidence of model suitability. They also help to explain some disappointing recurrent results. (AU)

Antecedentes: se explora la adecuación de un modelo “Dual” en el que ítems y personas son fuente de error de medida, evaluando como se espera que se comporten las puntuaciones en un test en términos de fiabilidad y validez cuando el modelo se cumple. Método: se derivan analíticamente predicciones respecto a: (a) el impacto del error en personas y en ítems en las estimaciones de fiabilidad y validez externa, así como en efectos techo esperados, (b) cambios en la fiabilidad marginal en grupos con diferente magnitud media de error individual, y (c) el fenómeno de la predictibilidad diferencial. Se incluyen dos estudios empíricos a efectos de ilustración y verificación empírica. Resultados: las predicciones concuerdan con la evidencia acumulada y con los principios de la teoría clásica del test. Sin embargo, la inclusión del parámetro de error individual permite llegar a nuevas explicaciones y predicciones. Conclusiones: la propuesta y resultados proporcionan nuevas fuentes de información en la medida de la personalidad, así como evidencia de la adecuación del modelo. También explican algunos resultados decepcionantes y recurrentes. (AU)

Is Correcting for Acquiescence Increasing the External Validity of Personality Test Scores? / ¿La Corrección por Aquiescencia Aumenta la Validez Externa de las Puntuaciones en Personalidad?

Background: Balanced scales control for acquiescence (ACQ) because the tendency of the respondent to agree with the positive items is cancelled out by the tendency to agree with opposite-pole items. When full balance is achieved, ACQ is not expected to affect external validity. Otherwise, attenuated estimates are expected to appear if no control methods such as Lorenzo-Seva & Ferrando’s (2009) are used. Method: Expected results were derived analytically. Subsequently, a simulation was carried out to assess (a) how ACQ impacted external validity and (b) how validity estimates behaved when ACQ was corrected. Two illustrative examples are provided. Results: A sizable number of items and/or high content loadings tended to decrease ACQ’s impact on validity estimates, making the empirical coefficient closer to its structural value. Furthermore, when scales were well balanced, the controlled and uncorrected scores were close to each other, and led to unbiased validity estimates. When the scales were unbalanced and no corrections were used, attenuated empirical validity coefficients inevitably appeared. Conclusions: Designing a well-balanced test or correcting for ACQ are the best ways to minimize attenuation in external validity estimation. (AU)

Antecedentes: construir escalas balanceadas permite controlar la aquiescencia (ACQ), haciendo que la tendencia del encuestado a estar de acuerdo con los ítems positivos se cancele con la tendencia a estar de acuerdo con los ítems del polo opuesto. En caso contrario, se esperarán estimaciones atenuadas de los coeficientes de validez externa en caso de no utilizar algún método de control (Lorenzo-Seva & Ferrando, 2009). Método: se llevó a cabo (a) un desarrollo analítico (b) una simulación para evaluar (a) el impacto de ACQ en la validez externa y (b) el comportamiento de las estimaciones de validez cuando se corrige por ACQ. Incluyendo finalmente dos ejemplos ilustrativos. Resultados: número alto de ítems y/o cargas altas en el factor de contenido tienden a disminuir el impacto de ACQ en las estimaciones de validez. Además, con escalas balanceadas por diseño, las diferencias entre las puntuaciones corregidas y no corregidas son menores, llevando a estimaciones de validez insesgadas. En escalas no balanceadas ni corregidas aparece una atenuación en el coeficiente de validez empírico. Conclusiones: diseñar pruebas balanceadas o corregir ACQ son las mejores maneras de minimizar la atenuación en la estimación de la validez externa. (AU)

Inventario IPIP-NEO: Estabilidad y Validez de Estructura Interna, Convergente y Concurrente en Muestras Argentinas / IPIP-NEO Inventory: Stability and Internal Structure, Convergent and Concurrent Validity in Argentinean Samples

Resumen Las principales herramientas de evaluación de la personalidad se construyeron a partir de las bases teóricas del modelo de los Cinco Factores. Particularmente, en Argentina, el IPIP-NEO es uno de los instrumentos, basados en este modelo, que presenta resultados satisfactorios en los estudios de consistencia interna (confiabilidad) y de validez mediante evidencia de estructura interna (análisis factorial exploratorio). El objetivo de este trabajo fue aportar nuevos estudios psicométricos a la escala utilizando una muestra heterogénea de argentinos (N = 499). Se estimó la estabilidad temporal de las puntuaciones a través del método test-retest y se realizó un estudio de validez de estructura interna mediante análisis factorial confirmatorio. Por otro lado, se aportó validez externa a través de estudios de convergencia, estudios de contraste de grupos divididos por sexo y edad, y un análisis test criterio mediante regresión múltiple con actividades recreativas. Los resultados sugieren que las puntuaciones que del IPIP-NEO son estables, con adecuados índices de confiabilidad y evidencias de validez externa, lo que indica que puede ser utilizado en muestras argentinas. Se planifica la elaboración de un baremo local y nuevos estudios de validez externa con implicancias clínicas y laborales.

Abstract The main personality assessment tools were built on the theoretical basis of the Five Factors model. Particularly, in Argentina, IPIP-NEO is one of the instruments, based on this model, that presents satisfactory results in the studies of internal consistency (reliability) and validity through evidence of internal structure (exploratory factor analysis). The aim of this work was to contribute new psychometric studies to the scale using a heterogeneous sample of Argentinean (N = 499). The temporal stability of the scores was estimated through the test-retest method and an internal structure validity study was performed by confirmatory factor analysis. On the other hand, external validity was provided through convergence studies, contrast studies of groups divided by sex and age, and a test criterion analysis through multiple regression with recreational activities. The results suggest that the IPIP-NEO scores are stable, with adequate reliability indices and evidence of external validity, indicating that it can be used in Argentine samples. The development of a local scale and new studies of external validity with clinical and labor implications are planned.

Análisis crítico de la literatura: un proceso elemental en el ejercicio de la práctica clínica basada en la evidencia / Critical analysis of literature: a basic process in exercising evidence based clinical practice

Para realizar una práctica clínica basada en la evidencia (PCBE), se requiere adquirir una serie de habilidades; entre ellas la búsqueda de literatura en forma sistemática y eficiente y la aplicación de reglas formales para la evaluación de la literatura encontrada. De este modo, se logra obtener de forma precisa la mejor evidencia disponible. Es así como, se podría resumir que el proceso de PCBE incluye varios pasos, uno de los cuales es el análisis crítico de la evidencia encontrada en la búsqueda de la literatura. Es fundamental comprender que no toda la información proveniente de revistas científicas es confiable o verdadera y por ende adecuada para ser instaurada como guía para la práctica clínica. Por tal razón, la información debe ser analizada de forma crítica; es decir, examinar cuidadosa y sistemáticamente la investigación para juzgar su validez y confiabilidad; o dicho de otra forma, su valor y relevancia en un contexto particular. El objetivo de este artículo fue jerarquizar la información existente respecto de estrategias para realizar un análisis crítico de la literatura científica y describir las herramientas más utilizadas con este fin.

To carry out clinical practice based on evidence (CPBE), it is necessary to acquire some skills, including systematic literature search and application of formal rules for evaluation of the literature found, thereby obtaining the best evidence available. Thus, one could summarize the CPBE process includes several steps, one of which is critical appraisal of the evidence found in literature search. It is essential to understand that not all the information from scientific journals is reliable or valid, and therefore suitable to be instituted in clinical practice. For this reason, the information must be critically analyzed; i.e. carefully and systematically examining research to judge its validity and reliability, or stated another way, its value and relevance in a particular context. The aim of this article is to hierarchy the existing information on strategies to realize critical analysis of the scientific literature and to describe the tools most commonly used for this purpose.

Análisis crítico de la literatura biomédica / Critical analysis of biomedical literature

Ante el gran número de publicaciones existentes y las que día a día incrementan el volumen total, generando un verdadero "bombardeo de información científica"; resulta un verdadero problema el escoger un estudio que responda a las inquietudes clínicas con un buen nivel evidencia. Para ello, se requiere seguir una metodología científica; definiendo y acotando de forma precisa el problema, para luego transformarlo en una pregunta contestable; aplicando posteriormente una estrategia de búsqueda en una fuente de información apropiada; para finalmente, analizar la información de forma crítica y resumir la evidencia encontrada. Para analizar la evidencia se han desarrollado guías de lectura o guías de usuario; de las que existen diversos formatos, pero que en términos generales dan las pautas necesarias para analizar de forma crítica los distintos tipos de artículos biomédicos según sea el escenario clínico al cual pertenezca. Estas herramientas evalúan tres aspectos fundamentales de un estudio: la validez interna, el impacto y la validez externa. Los aspectos antes mencionados se evalúan de forma práctica a través de ejemplo, aplicando una guía de usuario para artículos referentes a tratamiento a un estudio publicado en una prestigiosa revista científica.

Given the large number of publications, it is a problem to choose a study to respond to clinical problems with a good level of evidence. This requires following a scientific methodology, defining and limiting the problem, transforming it into a question, applying a search strategy in an appropriate source of information and finally, analyzing the information and summarizing the evidence found. Specific tools for analize the evidence had been develop. They are known as reading guides, Reader's Guides or Users' Guides, of which there are distinct formats, but in general, they give the necessary guidelines for the critical analysis of the different tipes of biomedical articles. These tools assess three key areas of study: internal validity, impact and external validity. The areas above mentioned are practically evaluated applying a user's guide to articles relating to treatment, as an example to a study published in a prestigious scientific journal.

Estudios experimentales sobre corrupción y el problema de la validez externa / Experimental studies on corruption and the external validity problem

El estudio de la corrupción (causas, consecuencias y medios efectivos para combatirla) es difícil debido a la naturaleza usualmente secreta del tipo de relaciones involucradas. Recientemente se han comenzado a realizar estudios experimentales sobre situaciones análogas a las de corrupción para complementar otros abordajes empíricos más tradicionales. El problema principal que se ha planteado sobre estos estudios es el de su validez externa, es decir, si los resultados que se obtienen en el laboratorio son extrapolables a situaciones de corrupción reales. Los objetivos de este artículo son, por un lado, resumir la metodología y los principales resultados de esta nueva área de investigación y, por otro, analizar el problema de su validez externa. Se concluye que si bien algunos resultados son alentadores, parece aconsejable una actitud cautelosa con respecto a la generalización de los mismos. Consecuentemente, se sugiere la necesidad de realizar más estudios empíricos para evaluar la validez externa del área.(AU)

Some authors conceive corruption as the misuse or abuse of public power for private benefit. Despite not being full agreement about the conceptual definition of the term, there is consensus that corruption is a widespread welfare-reducing phenomenon affecting institutions all over the world (Rose-Ackerman, 2006). The study of corruption (causes, consequences, and effective means of deterrence) is difficult because of the usually secret nature of the relationships involved. Despite the progress made in cross-country econometric analysis of corruption, on the level of the determinants of behaviour there is still lack of reliable knowledge about the factors that affect people’s corruptibility or the institutional features that affect the incidence of corruption (Abbink, 2006). The new field of experimental studies of corruption has recently emerged in an attempt to complement more traditional studies of corruption. In this paper, we first present a brief review of some of the most significant studies in this nascent area, in especial, we focus on introducing the methodology and main results of bribery experiments. In effect, although corruption is a phenomenon that can come in a variety of forms, the offer and acceptance of bribes has been the main focus of interest of this new experimental area. One of the simples ways in which bribery has been modeled in an experiment has been allowing couples of participants to play a Trust Game that can have negative consequences on the other couples (or on passive third parties, depending on the experiment; Abbink et al., 2002; Barr & Serra, 2009; Cameron et al., 2009). Negative externalities happen when the monetary transfer from the first to the second player (the bribe) is reciprocated by the second player choosing an option that favors the first player but harms others. Researchers have also experimentally implemented the possibility of a monetary punishment when the second player chooses the option with negative externalities (Abbink et al., 2002; Cameron et al., 2009). There are two goals that this new area of research pursues. First, researchers try to find out the micro-determinants of corrupt behavior. Second, researchers try to test the efficacy of different anticorruption policies. One of the main problems with experimental studies of corruption is their external validity, that is, the extent to which experimental results can be extrapolated to understand and combat corruption in real life. The second goal of this article is to discuss the external validity issue in experimental studies of corruption from particular examples from bribery studies. We mention two relevant aspects to assess the external validity of studies: on one hand, generalization among different populations, in which the idea is to evaluate the extent to which the same results persist when populations differ between experiments; on the other hand, situational generalization, in which the idea is to evaluate if results persist when the situation or the context varies. It has been argued (e.g., Levit & List, 2007) that the artificial features of the experimental setup (such as the use of abstract language, the detailed characterization of the corrupt transaction and the scrutiny of participants’ actions by researchers) make the extrapolation inference very problematic. We conclude that, although some results seem promissory (especially, very similar findings between laboratory and field experiments; Armantier & Boly, 2011), it seems recommendable to keep a cautious attitude toward the generalization of laboratory findings because there is not enough data on this field yet. Consequently, we stress that the problem of generalization can be approached empirically, and that it is necessary to have more studies which aim at evaluating the external validity of experiments in an area which pretends to be relevant to applied issues.(AU)

Estudios experimentales sobre corrupción y el problema de la validez externa / Experimental studies on corruption and the external validity problem

El estudio de la corrupción (causas, consecuencias y medios efectivos para combatirla) es difícil debido a la naturaleza usualmente secreta del tipo de relaciones involucradas. Recientemente se han comenzado a realizar estudios experimentales sobre situaciones análogas a las de corrupción para complementar otros abordajes empíricos más tradicionales. El problema principal que se ha planteado sobre estos estudios es el de su validez externa, es decir, si los resultados que se obtienen en el laboratorio son extrapolables a situaciones de corrupción reales. Los objetivos de este artículo son, por un lado, resumir la metodología y los principales resultados de esta nueva área de investigación y, por otro, analizar el problema de su validez externa. Se concluye que si bien algunos resultados son alentadores, parece aconsejable una actitud cautelosa con respecto a la generalización de los mismos. Consecuentemente, se sugiere la necesidad de realizar más estudios empíricos para evaluar la validez externa del área.

Some authors conceive corruption as the misuse or abuse of public power for private benefit. Despite not being full agreement about the conceptual definition of the term, there is consensus that corruption is a widespread welfare-reducing phenomenon affecting institutions all over the world (Rose-Ackerman, 2006). The study of corruption (causes, consequences, and effective means of deterrence) is difficult because of the usually secret nature of the relationships involved. Despite the progress made in cross-country econometric analysis of corruption, on the level of the determinants of behaviour there is still lack of reliable knowledge about the factors that affect people’s corruptibility or the institutional features that affect the incidence of corruption (Abbink, 2006). The new field of experimental studies of corruption has recently emerged in an attempt to complement more traditional studies of corruption. In this paper, we first present a brief review of some of the most significant studies in this nascent area, in especial, we focus on introducing the methodology and main results of bribery experiments. In effect, although corruption is a phenomenon that can come in a variety of forms, the offer and acceptance of bribes has been the main focus of interest of this new experimental area. One of the simples ways in which bribery has been modeled in an experiment has been allowing couples of participants to play a Trust Game that can have negative consequences on the other couples (or on passive third parties, depending on the experiment; Abbink et al., 2002; Barr & Serra, 2009; Cameron et al., 2009). Negative externalities happen when the monetary transfer from the first to the second player (the bribe) is reciprocated by the second player choosing an option that favors the first player but harms others. Researchers have also experimentally implemented the possibility of a monetary punishment when the second player chooses the option with negative externalities (Abbink et al., 2002; Cameron et al., 2009). There are two goals that this new area of research pursues. First, researchers try to find out the micro-determinants of corrupt behavior. Second, researchers try to test the efficacy of different anticorruption policies. One of the main problems with experimental studies of corruption is their external validity, that is, the extent to which experimental results can be extrapolated to understand and combat corruption in real life. The second goal of this article is to discuss the external validity issue in experimental studies of corruption from particular examples from bribery studies. We mention two relevant aspects to assess the external validity of studies: on one hand, generalization among different populations, in which the idea is to evaluate the extent to which the same results persist when populations differ between experiments; on the other hand, situational generalization, in which the idea is to evaluate if results persist when the situation or the context varies. It has been argued (e.g., Levit & List, 2007) that the artificial features of the experimental setup (such as the use of abstract language, the detailed characterization of the corrupt transaction and the scrutiny of participants’ actions by researchers) make the extrapolation inference very problematic. We conclude that, although some results seem promissory (especially, very similar findings between laboratory and field experiments; Armantier & Boly, 2011), it seems recommendable to keep a cautious attitude toward the generalization of laboratory findings because there is not enough data on this field yet. Consequently, we stress that the problem of generalization can be approached empirically, and that it is necessary to have more studies which aim at evaluating the external validity of experiments in an area which pretends to be relevant to applied issues.